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Abstract. Incremental SAT and QBF solving potentially yields improvements 
when sequences of related formulas are solved. An incremental application is 
usually tailored towards some specific solver and decomposes a problem into 
incremental solver calls. This hinders the independent comparison of different 
solvers, particularly when the application program is not available. As a remedy, 
we present an approach to automated benchmarking of incremental SAT and 
QBF solvers. Given a collection of formulas in (Q)DIMACS format generated 
incrementally by an application program, our approach automatically translates 
the formulas into instructions to import and solve a formula by an incremental 
SAT/QBF solver. The result of the translation is a program which replays the incre¬ 
mental solver calls and thus allows to evaluate incremental solvers independently 
from the application program. We illustrate our approach by different hardware 
verification problems for SAT and QBF solvers. 


1 Introduction 

Incremental solving has contributed to the success of SAT technology and potentially 
yields considerable improvements in applications where sequences of related formulas 
are solved. The logic of quantified Boolean formulas (QBF) extends propositional logic 
(SAT) by explicit existential and universal quantification of variables and lends itself for 
problems within PSPACE. Also for QBFs, incremental solving has been successfully 
applied in different domains 04171111121 . 

The development of SAT and QBF solvers has been driven by competitive events 
like the SAT Competitions, QBF Evaluations (QBFEVAL), or the QBF Galleries. These 
events regularly result in publicly available benchmarks submitted by the participants 
which help to push the state of the art in SAT and QBF solving. In the past, the focus 
was on non-incremental SAT solving, and the evaluation of incremental solvers does not 
readily benefit from competitions and available benchmark collections. 

Benchmarking incremental solvers requires to solve a sequence of related formulas. 
To this end, the formulas must be incrementally imported to the solver and solved by 

* This work was supported by the Austrian Science Fund (FWF) under grant S11409-N23. 
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means of API calls. The API calls are typically generated by an application program, 
like a model checker or a formal verihcatlon or planning tool, for example, which tackles 
a problem by encoding it incrementally to a sequence of formulas. In order to compare 
different incremental solvers on that sequence of formulas, the solvers must be tightly 
coupled with the application program by linking them as a library. Hence benchmarking 
of incremental solvers relies on the application program used to generate the sequence 
of formulas which, however, often is not available. Even if the application program is 
available, it has to be adapted to support different solvers, where each solver might come 
with its own API. Further, the same sequence of formulas must be generated multiple 
times by the application program to compare different solvers. 

To remedy this situation, we present an approach to automated benchmarking of 
incremental SAT and QBE solvers which decouples incremental SAT/QBF solving from 
incremental generation of formulas using an application program. This is achieved by 
translating a sequence of related CNFs and QBFs In prenex CNF (PCNF) into API calls 
of incremental solvers. Such a sequence might be the output of an application program 
or it was taken from existing benchmark collections. The formulas are then syntactically 
analyzed and instructions to incrementally import and solve them are generated. For 
CNFs, the instructions are function calls in the IPASIR API, which has been proposed 
for the Incremental Library Track of the SAT Race 2015p]For PCNFs, the instructions 
correspond to calls of the API of the QBF solver DepQBFl^which generalizes IPASIR 
and allows to update quantiher prehxes. The result of translating a sequence of formulas 
to solver API calls is a standalone benchmarking program which replays the incremental 
solver calls. Any incremental SAT/QBF solver supporting the IPASIR API or its QBF 
extension as implemented in DepQBF can be integrated by simply linking it to the 
program. This allows to compare different solvers independently from an application. 

In some applications, the sequence of formulas depends on the used solver, e.g., 
if truth assignments are used to guide the process. Even then, our approach allows 
to compare different incremental solvers on the fixed sequences generated with one 
particular solver. However, then It is important to note that this comparison is limited to 
this particular hxed sequence, it would be unfair to conclude something about the perfor¬ 
mance of the solvers would they have been genuinely used within the application. This 
problem occurs also in sequences of formulas which are already present in benchmark 
collections. For experiments in this paper, we only considered applications where the 
sequences of generated formulas do not depend on intermediate truth assignments. 

As our approach is also applicable to already generated formulas that are part of 
existing benchmark collections, such collections become available to developers of 
incremental solvers. Furthermore, comparisons between solvers In Incremental and non- 
Incremental mode are made possible. In addition, since the input for the benchmarking 
program describes only the differences between consecutive formulas, we obtain a 
quite succinct representation of incremental benchmarks. Our approach to automated 
benchmarking of incremental SAT and QBF solvers underpins the goal of the Incremental 
Library Track of the SAT Race 2015. We have generated benchmarks and submitted 
them to this competition. 

’http://baldur.iti.kit.edu/sat-race-2015/ 

^http://lonsing.github.io/depqbf/ 
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2 Background 

We consider propositional formulas in CNF and identify a CNF with the set of its clauses. 
A sequence a — [Fi,... ,Fn) of formulas represents the formulas that are incrementally 
generated and solved by an application program. A QBF ij} = P.F in prenex CNF 
(PCNF) extends a CNF F by a quantifier prefix P. The prefix P = Qi,... , of a 
QBF is a sequence of pairwise disjoint quantified sets Qi. A quantified set Q is a set 
of variables with an associated quantifier quant{Q) G {3, V}. We consider only closed 
PCNFs. For adjacent quantified sets Qi and Qi+i, quant{Qi) quant{Qi+i). Given a 
prefix P = Qi, ..., Qn, index i is the nesting level of Qi in P. 

Our automated benchmarking approach is based on solving under assumptions 061 
as implemented in modern SAT 11191131 and QBF solvers IIIOII11121 . When solving a 
CNF under assumptions, the clauses are augmented with selector variables. Selector 
variables allow for temporary variable assignments made by the user via the solver API. 
If the value assigned to a selector variable satisfies the clauses where it occurs, then these 
clauses are effectively removed from the CNF. This way, the user controls which clauses 
appear in the CNF in the forthcoming incremental solver run. The IPASIR API proposed 
for the Incremental Library Track of the SAT Race 2015 consists of a set of functions for 
adding clauses to a CNF and handling assumptions. A disadvantage of this approach is 
that the user has to keep track of the used selector variables and assumptions manually. 

For incremental QBF solving, additional API functions are needed to remove quanti¬ 
fied sets and variables from and add them to a prefix. For QBF solvers, we generate calls 
in the API of DepQBF which generalizes IPASIR by functions to manipulate quantifier 
prefixes. Additionally, it allows to remove and add clauses in a stack-based way by 
push/pop operations where selector variables and assumptions are handled internal to 
the solver and hence are invisible to the user fiOll . For details on the IPASIR and DepQBF 
Interfaces, we refer to the respective webpages mentioned in the introduction. 

3 Translating Related Formulas into Incremental Solver Calls 

We present the workflow to translate a given sequence a = {ipi,... ,'4’n) of related 
(P)CNFs into a standalone benchmarking program which calls an integrated solver via 
its API to incrementally solve the formulas from tpi up to ipn'- 

1. First, the formulas in a are analyzed and the syntactic differences between each 
ipi and V'i+i are identified. This includes clauses and quantified sets that have to 
be added or removed to obtain ipi+i from ipi. Also, variables may be added to or 
removed from quantified sets. For CNFs, the prefix analysis is omitted. 

2. The differences between the formulas identified in the first step are expressed by 
generic update instructions and are written to a file. A clause set is represented as a 
stack which can be updated via push and pop operations. The update instructions 
for quantifier prefixes are adding a quantified set at a nesting level and adding new 
variables to quantified sets already present in the prefix. Unused variables are deleted 
from the prefix be the solver. 

3. Files that contain generic update instructions are then interpreted by a benchmarking 
program which translates them into calls of the IPASIR API (for CNFs) or QBF 
solver calls (for PCNFs). For the latter, calls of DepQBF’s API are generated. 
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The benchmarking program is standalone and independent from the application program 
used to generate a. It takes the files containing the generic update instructions as the 
only input. Multiple solvers may be integrated in the benchmarking program by linking 
them as libraries. Files containing the update instructions can serve as standardised 
benchmarks for incremental SAT and QBF solvers. 

Analyzing CNFs. The algorithm to analyze sequences a = (Fi,..., F„) of clause sets 
relies on a stack-based representation of Fi which allows for simple deletion of clauses 
that have been added most recently. A clause c which appears in some Fi and is removed 
later at some point to obtain Fj with i < j < nis called volatile in Fi. A clause which 
appears in some Fi for the first time and also appears in every Fj with i < j < n and 
hence is never deleted is called cumulative in Fi. 

The algorithm to analyze sequence cr identifies volatile and cumulative clauses in all 
clause sets in tr. Cumulative clauses are pushed first on the stack representing the current 
clause set because they are not removed anymore after they have been added. Volatile 
clauses are pushed last because they are removed at some point by a pop operation 
when constmcting a later formula in a. For illustration, consider the following sequence 
a = {Fi,..., F 4 ) of clause sets Fi along with their respective sets Ci of cumulative 
clauses and sets Vi of volatile clauses; 


Fi = {Ci,C2,Wl} 

F 2 = {ci,C2,C3,Vi,V2} 

F 3 = {C1,C2,C3,C4,V1,V3} 
F4 = {Ci, C2, C3, C4, C5} 


Ci={Ci,C2} V4={V4} 

C2 = {cs} V2 = {vi,l’2} 

C3 = {C4} V'3 = {V1,V3} 

C4 = {cs} V4 = 0 


After the sets of cumulative and volatile clauses have been identified for each Fi, the 
clause sets can be incrementally constructed by means of the following operations on the 
clause stack: adding a set C of clauses permanently to a formula by add(C'), pushing 
a set C of clauses on the stack by push(C), and popping a set of clauses from the 
stack by pop(). The sequence tr = (Fi,..., F4) from the example above is generated 
incrementally by executing the following stack operations: 


add(Ci) push(Vi) 


pop() add(C2) push(V2) 

popQ add(C3) push(V3) 

pop() add(C4) push(V4) 


Note that the above schema of stack operations generalises to arbitrary sequences of 
clause sets, i.e., we need at most one push, one add, and one pop operation in each step, 
provided that the clauses have been classified as volatile or cumulative before. 

The algorithm for identifying cumulative and volatile clauses in a sequence of clause 
sets appears as Algorithm[T] For SAT solvers supporting the IPASIR API, stack frames 
for volatile clauses pushed on the clause stack are implemented by selector variables. 
Our current implementation of the benchmarking program includes DepQBF as the only 
incremental QBF solver which supports push/pop operations natively via its API ifTOll . 
Note that the relevant part of the input that potentially limits scalability of Algorithm[T]is 
the number of variables and clauses in the formulas. The number of formulas is usually 
relatively low. The operations on clause sets are implemented such that set intersection 


Automated Benchmarking of Incremental SAT and QBF Solvers 


5 


Input : Clause sets Ti, F 2 , • • •, -F’n (at least two sets are required) 

Output : Cl,, Cn (sets of cumulative clauses to be added) 

Vi,..., (sets of volatile clauses to be pushed or popped) 

1 Vr^Fr\F^\ Ci^Fi\I/i; 

2 for 2 ^ 2 to n — 1 do 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 Cn ^— Fn \ Fn-v, V„ <— 0; 

Algorithm 1: Identifying cumulative and volatile clauses. 


Vi^Fi\Fi+i- 
Ci^ {Fi\Fi-i)\V.- 

foreach c G Vi D Fi-i do 
for j t— 1 to i — 1 do 
if c G Cj then 

{c}; 

for fc = j to i — 1 do 

L Vk^VkUic}- 

break; 


and difference are in 0 {m ■ logm), searching an element is in 0 {m), and adding or 
deleting elements are in 0(1), where m is the maximal number of clauses in any formula. 

Analyzing PCNFs. For sequences of QBFs, additionally the differences between quanti¬ 
fier prefixes must be identified. Two quantified sets Q and Q' are matching iff QFQ' 
Prefix R is update-compatible to prefix S iff all of the following conditions hold; (i) for 
any quantified set of R, there is at most one matching quantified set in S'; (ii) if P is a 
quantified set of R and Q is a matching quantified set in S, then quant{P) = quant{Q)\ 
and (iii) for any two quantified sets Pi and P 2 in S with matching quantified sets Qi 
and Q 2 in R, respectively, if the nesting level of Pi is less than the nesting level P 2 , then 
the nesting level of Qi is less than the nesting level of Q 2 . 

The instructions to update quantifier prefixes are adding a quantified set at a given 
nesting level or adding a variable to a quantified set at a given nesting level. Update 
compatibility between prefixes R and S guarantees that there is a sequence of instructions 
to turn R into S after unused variables and empty quantified sets have been deleted by 
the QBF solver. In particular. Condition (i) guarantees that there is no ambiguity when 
mapping quantified sets from the prefixes, (ii) expresses that quantifiers cannot change, 
and (iii) states that quantified sets cannot be swapped. The algorithm to generate update 
instructions first checks if two quantifier prefixes R and S are update-compatible. If this 
is the case, then update instructions are computed as illustrated by Algorithm]^ 


4 Case Studies 

In this section, we showcase our approach using different hardware verification problems 
for both SAT and QBF solvers. Benchmark problems consist of sequences of formulas 
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Input : Prefix R and S (R has to be update-compatible to S) 

Output : Instructions to update Rto S 

1 n < — 0; m <— 0; 

2 foreach quantified set Q in S from left to right do 

3 if Q has a matching quantified set M in R then 

4 m <— n + nesting level of M in R; 

5 print “Add literals Q\M to quantified set at nesting level m.”\ 

6 else 

7 n <— n + 1; 

8 m ■(— m + 1; 

9 print "Add quantified set Q at nesting level m.”; 

Algorithm 2: Generating update instructions for quantifier prefixes. 

Table 1. Summary of different SAT solvers on hardware verification problems. 



#problems 

MiniSAT 

PicoSAT 

Lingeling 

DepQBF 

BMC problems unrolled by 50 steps 

11 

284/7 

216/3 

276/7 

190/1 

BMC problems unrolled by 100 steps 

28 

905 / 14 

754/4 

872/ 19 

491/2 


that were either generated by a model-checking tool or that were taken from existing 
benchmark collections where the original application is not available. 

SAT: Bounded-Model Checking for Hardware Verification. We consider benchmarks 
used for the single safety property track of the last Hardware Model Checking Com¬ 
petition (HWMCC 2014^^ Based on the CNFs generated by the BMC-based model 
checker aigbmcj^ we use our tools to generate incremental solver calls and compare 
different SAT solvers that implement the IPASIR interface. We used the SAT solvers 
MiniSAT (v.220) Q, PicoSAT (v.961) and Lingeling (v.ayv) ||3l as well as the 
QBF solver DepQBF (v.4) for the considered problems. All experiments were performed 
on an AMD Opteron 6238 at 2.6 GHz under 64-bit Linux with a time limit of 3600 
seconds and a memory limit of 7 GB. 

Tablesummarises the results. For each solver and problem class, numbers m / n 
mean that m formulas in total were solved within the time limit, and n is the number 
of problems where the maximal number of formulas among all other solvers could 
be solved. For example, the first line summarises the results for BMC problems that 
were unrolled by 50 steps. There are 11 problems in this class, thus 550 formulas in 
total. From these formulas, MiniSAT could solve 284 formulas, and for 7 out of 11 
problems, no other solver could solve more formulas than MiniSAT. Not surprisingly, all 
SAT solvers outperform the QBF solver DepQBF but there are few cases where DepQBF 
can compete. MiniSAT solves most formulas in total while Lingeling dominates on 
most benchmarks. More detailed experimental results can be found in the appendix. 
The average time for our analyzing algorithm was 522 seconds. The number of clauses 
in the original sequences ranged from 2.3 to 56.3 million with an average of around 

^ http://fmv.jku.at/hwraccl4cav/ 

* Part of the AIGER package (http: / / fmv. jku. at/aiger /) 
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19 million clauses. The inputs for the benchmarking program that represent only the 
update instructions comprise only 1.2 million clauses on average which shows that we 
obtain a quite compact representation of incremental benchmarks. We have submitted 
all problems from Table[2to the Incremental Library Track of the SAT Race 2015. 

QSAT: Partial Design Problems. 
To illustrate our approach in the 
context of QBF solving, we con¬ 
sider the problem of verifying 
partial designs, i.e., sequential 
circuits where parts of the spec¬ 
ification are black-boxed. In re¬ 
cent work nma, the question 
whether a given safety property 
can be violated regardless of 
the implementation of a black¬ 
box has been translated to QBFs 
which are solved incrementally 
by a version of the QBF solver 
QuBE 0. Benchmarks are avail¬ 
able from QBFLIB]^ however 
neither the solver used in rmrii 
nor the application program used 
to generate sequences of QBFs 
are publicly available. Marin et al. im introduced two encoding strategies: forward 
incremental and backward incremental reasoning. In a nutshell, the quantiher prehx is 
always extended to the right in the former approach, while it is extended to the left in the 
latter approach. Both strategies yield the same sequences of formulas up to renaming ca. 
We used the publicly available instances from the forward-incremental encoding without 
preprocessing to evaluate DepQBF. Instances from the backward-incremental approach 
are not publicly available. 

Table 1^ shows the comparison between QuBE and DepQBF. Runtimes are in seconds, 
k is the index of the hrst satisfiable formula, TO and MO refer to a timeout and memout, 
respectively. The maximal rantime of Algorithm[T]and[^was 95 seconds. Runtimes for 
QuBE in Table|^are the ones reported in ifTTIl . There, experiments were carried out on 
an AMD Opteron 252 processor running at 2.6 GHz with 4GB of main memory and 
a timeout of 7200 seconds. Experiments for DepQBF were performed on a 2.53 GHz 
Intel Core 2 Duo processor with 4GB of main memory with OS X 10.9.5 installed. 
Thus runtimes are not directly comparable because experiments were carried out on 
different machines, they give, however, a rough picture of how the solvers relate. Like 
QuBE, DepQBF benehts from the incremental strategy on most instances. The backward- 
incremental strategy is clearly the dominating strategy for QuBE. A quite eye-catching 
observation is that forward-incremental solving, while hardly improving the performance 
of QuBE compared to the non-incremental approach, works quite well for DepQBF. 

^ http://WWW.qbflib.org 


Table 2. QBF solvers on incomplete design problems. 


Benchmark 

k 

non-incremental 

QuBE DepQBF 

incremental 

QuBE (fwd) QuBE (bwd) DepQBF 

enc04 

17 

3 

3 

3 

2 

1 

enc09 

17 

7 

5 

7 

4 

3 

encOl 

33 

31 

17 

28 

24 

5 

enc03 

33 

33 

16 

289 

28 

27 

enc05 

33 

64 

24 

61 

46 

7 

enc06 

33 

29 

26 

28 

24 

10 

encOV 

33 

75 

16 

76 

69 

5 

enc08 

33 

108 

16 

110 

79 

5 

enc02 

65 

271 

106 

TO 

269 

175 

tlcOl 

132 

26 

68 

133 

130 

17 

tlc03 

132 

24 

160 

8 

8 

17 

tlc04 

132 

769 

2196 

1204 

27 

25 

tlc05 

152 

1330 

4201 

2057 

38 

34 

tlc02 

258 

MO 

TO 

MO 

98 

1908 
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5 Conclusion 

We presented an approach to automated benchmarking of incremental SAT and QBF 
solvers by translating sequences of formulas into API calls of incremental SAT and 
QBF solvers executed by a benchmarking program. Several incremental solvers may be 
tightly integrated into the benchmarking program by linking them as libraries. Thus, we 
decouple the generation of formulas by an application from the solving process which is 
particularly relevant when application programs are not available. Additionally, we make 
sequences of formulas which already exist in public benchmark collections available for 
benchmarking and testing. We illustrated our approach to automated benchmarking of 
incremental SAT and QBF solvers on instances from hardware verification problems. To 
improve the performance of incremental QBF solving on these problems, we want to 
integrate incremental preprocessing into DepQBF. As shown in 111 1I12II . preprocessing 
potentially improves the performance of incremental workflows considerably. 
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A Correctness of Algorithms 1 and 2 

Theorem 1. Algorithm^is totally correct with respect to the precondition that a — 
{Fi ,..., Fn) is a sequence of sets of clauses with n> 2 and the postcondition that any 
Ci, 1 < i < n, contains the cumulative clauses of Fi in a, and any Vi, 1 < i < n, 
contains the volatile clauses of Fi in a. 

Proof Clearly, Algorithm[T]tertninates on each input. 

We show that the condition that any Cj, 1 < j < i, contains the cumulative clauses 
of Fj in the subsequence = {Fi,Fi) of a, and any Vj, 1 < j < i, contains the 
volatile clauses of Fj in ai is an invariant of the main loop (at Line 2). The invariant 
together with i = n implies the postcondition as C„ always contains those clauses that 
are in Fn but not in Fn-i, and Vn always equals the empty set (Line 12). Likewise, the 
precondition implies the invariant since after Line 1, Vi contains all clauses of Fi that 
are not in F 2 and which are thus volatile in Fi in the sequence Fi,F 2 , and Ci contains 
all clauses which are in Fi and F 2 and which are hence cumulative in Fi in the sequence 
Fi, F 2 . 

It remains to show that if the invariant holds for some i, 2<i<n— lat Line 3, 
then it holds for i +1 after executing Lines 3-11. After Line 3, Vi contains all the clauses 
that are volatile in Fi in 0 -^+ 1 . Likewise, after Line 4, Ci contains all the clauses that 
are in L) but not in Li-i and which are not volatile in Fi, that is, which are cumulative 
in Fi in ai+i. Note that if a clause c is volatile in some Fj, j < i, in ai, then c is also 
volatile in Fj in On the other hand, if a clause is cumulative in Fj, it can be the 
case that c becomes volatile in 17^+1 if c ^ Ti+i- Hence, it is possible that clauses that 
were previously classified as cumulative need to be reclassified. 

We make use of the following claim: After Line 4, a clause c is in Vi n Fi_i iff, 
for some j < i, Cj contains a clause c that is volatile in ai+i. This claim is proven as 
follows: Assume that for some j < i, Cj contains a clause c that is volatile in cTi+i. As 
the invariant holds for i, c G Fk, for all j < A: < i but c ^ Ti+i and thus c GVi. Clearly, 
c G Vi Fi-i. On the other hand, assume some clause c is in Vi n Li-i. Clearly, c G Vi 
implies c G Fi. Hence, as the invariant holds for i, c G Cj, for some j < i, and, since 
c G Vi, c is volatile in Fj in 

By virtue of the above claim, Vi C Li-i contains precisely those clauses which 
need to be reclassified as volatile. After Lines 6-11, for each c G Vi D Li_i, the first 
(and only) Cj with c G Cj is found, and c is removed from Cj and added to all V), 
j < I < i — 1. Hence, after Lines 3-11, the invariant holds for i + 1. □ 

Algorithm 1^ works as follows. For each quantified set q of S, either it has one 
matching set q' in R (Lines 4 and 5) or it does not have a matching quantified set in 
R (Lines 7,8, and 9). In the former case, we need to add the atoms in q to q' if they 
are not already there (Line 5). In the latter case, we need to add the entire set q to the 
prefix (Line 9). Adding atoms and quantified sets is always done at the right nesting level 
m. We store in n the number of new quantified sets that have been added. At Line 5, 
when adding atoms to a matching quantified set, m is the nesting level of the matching 
quantified set in R plus the number n of previously added unmatched quantified sets. At 
Line 9, when adding an entire quantified set, m is the nesting level of the quantified set 
that was modified last plus one. 
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Table 3. Detailed results for Table[^ SAT solvers on hardware verification problems. 



MiniSAT 

PicoSAT 

Lingeling 

DepQBF 

6s393r 

15 

15 

14 

10 

6s394r 

27 

23 

22 

16 

6s514r 

17 

16 

16 

14 

arbiOsOS 

12 

12 

13 

11 

arbiOslfi 

17 

17 

17 

17 

arbixsOS 

9 

10 

10 

10 

cuabq2f 

28 

22 

40 

19 

cuabq2mf 

77 

46 

65 

20 

cuabq4f 

21 

22 

26 

15 

cuabq4mf 

25 

23 

40 

16 

cuabqSf 

22 

21 

26 

19 

cubak 

83 

43 

55 

28 

cufq2 

101 

88 

83 

21 

cugbak 

38 

32 

42 

21 

cuhanoilO 

35 

34 

35 

17 

cujcl2 

47 

46 

38 

15 

cuniml 

22 

21 

23 

19 

cunim2 

22 

21 

22 

18 

cuoml 

14 

13 

14 

10 

cuom2 

14 

13 

15 

10 



MiniSAT 

PicoSAT 

Lingeling 

DepQBF 

cuom3 

15 

14 

16 

10 

cuptsl4 

34 

32 

37 

25 

cuptsl5 

35 

32 

37 

27 

cuptsl6 

35 

34 

37 

24 

cutarbl6 

52 

37 

48 

30 

cutfl 

44 

32 

35 

21 

pdtfifoltoO 

16 

16 

16 

13 

pdtpmsdcl6 

28 

19 

30 

15 


BMC problems unrolled by 100 steps. 


6sl88 

39 

34 

36 

33 

6s24 

25 

23 

27 

20 

6s270bl 

51 

13 

51 

11 

arbi0s32p03 

32 

32 

33 

32 

arbixsl6p03 

16 

16 

16 

16 

bmhanlfl 

29 

21 

29 

19 

bobpcihm 

17 

15 

16 

11 

bobsmvhd3 

14 

11 

14 

10 

cufql 

45 

33 

37 

23 

cujcl28 

7 

8 

7 

7 

cujc32 

9 

10 

10 

8 


BMC problems unrolled by 100 steps. 


BMC problems unrolled by 50 steps. 



















